Issue: Exchange VSS Writer with Symantec Backup Exec 2012

I've recently spent some time working with different backup software products, and came across a particular issue whilst using Symantec Backup Exec 2012. The issue was that the Microsoft Exchange VSS Writer was not stable, and this was causing backups to fail.

If you require additional IT support, why not look through our collection of IT solution services.

Check Microsoft Exchange Writer status

The backups were failing because of an instability in the VSS Writer. Event ID 9607 was listed. If you experience the same issue, this is the error you might see; "Microsoft Exchange 2003/2007/2010 backup fails with an error "E000FED1: A failure occurred querying the Writer status"

This is how to find the status of the Microsoft Exchange Writer.

1. Click Start followed by Run.
2. Type cmd and click enter.
3. Once the command prompt box has been opened type vssadmin list writers

This command will list all of the Volume Shadow Copy Service writers in use; the one you will need to locate is the "Microsoft Exchange Writer". You will notice whether it is working or not by looking at the state: in this instance it had failed.

Normally a standard reboot of the server will fix this issue, but in my case a reboot didn't work, and so we restarted the Microsoft Exchange Information Store service.

Restart Microsoft Exchange Information Store

To do this, complete the following steps;

1. Click Start followed by Run.
2. Type cmd and click enter.
3. Type services.msc and click enter.
4. In the list of services shown, locate Microsoft Exchange Information Store and click on it, and in the top left you will see Start and Restart.

5. Once this has been completed and the service has fully restarted you can repeat the steps for checking Microsoft Exchange Writer status and you should see that the writer has become stable.

Once this has been completed and you see the writer in a stable condition, one thing to make sure is that the Exchange database doesn't dismount itself, as in some cases when performing these actions it was dismounting the Exchange database meaning we had to open the Exchange Management Console and remount it.

To check this you must open up Exchange and on the left hand side expand Organisation Configuration, select Mailbox and then under the Database Management tab on the mailbox database ensure it says "Mounted", if not you can simply right click and remount the database.

At this point in conversation with Symantec we were advised to re-run the backups again and so I would suggest trying to run a full backup followed by the incremental and see what the results are.

If the backups are still failing then the next thing to try is to make sure that circular logging is disabled.

How to Disable Circular Logging in Exchange 2007/2010

When circular logging is enabled, log files that have already been dedicated to the database are overwritten, preventing the build-up of logs. The log files are overwritten whether or not a full or incremental backup has been ran and a history of previous logs since the last full or incremental backup are not kept. This is why, when circular logging is enabled, differential and incremental backups of the Exchange databases cannot be performed (since these types of backup rely on a complete history of logs).

1. Open the Exchange Console
2. In the console tree, navigate to Organisation Configuration then Mailbox
3. In the action pane, under the database name, click Properties
4. Click on the Maintenance tab and then uncheck "Enable circular logging"

Once this has been done and to put the changes into effect, you must Restart the Microsoft Exchange Information Store service.

This can also be done using the Exchange Management Shell, using the following command:
Set-MailboxDatabase -Identity "Database Name" -CircularLoggingEnabled $false

To keep the pattern and flow of the diagnostic process consistent Symantec advise that you try and run another backup (full backup first, followed by the incremental).

Flushing the Exchange Transaction Logs

After advice from Symantec about switching off circular logging and re-running the backups in expectance of a pass, what we actually realised was the Exchange transaction logs that circular logging had been using weren't being created as they should have been and so since these types of backups rely on complete history of logs this was never going to work. This led us onto the next step of flushing out the current transaction logs in order for the new ones to be created. This would mean that when Symantec was looking for the history of logs it could see ones that were being created properly.

There are two ways of completing this process. Firstly, you can either simply move the database path itself so the logs would be going into a completely new folder, or secondly if you need to keep the logs for reference then the other option is to just recreate the folder that the logs are currently going into. An important thing to note is that, whenever you are doing anything related to the logs and something that is happening live, then in all cases the database would need to be dismounted. This is the exact same process as remounting the database but you would just be dismounting instead. It would be advisable to do this out of work hours, as this will result in the server temporarily not receiving emails.

Moving the Exchange Database Path

Before taking this step you must dismount the mailbox database as per the previous step.
To move the full mailbox database path you must:

1. Open the Exchange Management Console
2. Expand Organisation Configuration and select Mailbox
3. You must then select the primary database which should be named Mailbox Database unless this has been renamed
4. On the action pane at the bottom you will then notice the option to Move Database Path, select this. (NOTE: It is important that you don't have the Database Path and the Log Folder path pointing to the same location, as this will create an extremely large folder, meaning that if ever the database needs to be restored you will find yourself in a very sticky situation as Exchange when restoring will get confused as to what it is looking for and what files it was trying to restore.)
5. Once you have the location of the Database Path and the Log Folder Path, this is where you will need to create a new folder in the same location as the current Database Path, remembering to keep the Mailbox Database.edb file and the Log files separate

This process will dismount the Mailbox database and should be conducted out of hours.

These are the locations by default, although if your aim is to create a new folder for the Database Path you will need to move up a level in the Log Folder Path.

From here: C:Program FilesMicrosoftExchange ServerV14MailboxMailbox Database

To here: C:Program FilesMicrosoftExchange ServerV14Mailbox (the new folder will be created here)

Currently it is set to C:Program FilesMicrosoftExchange ServerV14MailboxMailbox Database.

To make things easy I have just renamed the Log Folder Path and put on the end .NEW so it looks like this.

In order to make the backups run more reliably, we attempted to split the backups to ensure that the data required to backup was being backed up alongside the Exchange Database.

The first backup job is a full backup of the Exchange Database, running once every day at a set time. The second backup job runs a full backup and an incremental backup of all the data on the server with an exclusion set for Exchange. It's very important to ensure that you exclude the Exchange Database as otherwise the incremental backup will fail. The schedule for these backups was decided based on server usage.

It should look something like this.

Although there is no official solution yet, hopefully this article is useful in giving you a variety of things to try.

If you have had similar issues or there is anything you can suggest to add to this post then please feel free to leave a comment! In the meantime if I find a definite fix I will add it - so keep an eye out for updates.